N95- 15980 


AN APPLICATIONS-ORIENTED APPROACH 
TO THE DEVELOPMENT OF VIRTUAL ENVIRONMENTS 

Michael X. Crowe 
GreyStone Technology, Inc. 

15010 Avenue of Science, Suite 200 
San Diego, CA 92128 
(619) 675-7800 
mxcrowe@gstone.com 


ABSTRACT 

The field of Virtual Reality (VR) is diverse, ranging in scope from research into fundamental 
enabling technologies to the building of full-scale entertainment facilities. Due to the multi- 
faceted nature of this field and complicated by excessive media attention and interpretation, the 
concept of virtual reality means many things to many people. Ideally, a definition of VR should 
derive from how this technology can provide solutions to existing challenges in building 
advanced human-computer interfaces. The measure of success for this technology lies in its 
ability to enhance the assimilation of complex information, whether to aid in difficult decision- 
making processes, or to recreate real experiences in a compelling way. This philosophy and the 
virtual environment development process employed by the engineers and artists at GreyStone 
Technology, Inc. is described using an example from a VR-based advertising project. The 
common and unique elements of this example are explained, though the fundamental 
development process is the same for all virtual environments that support information transfer. 
In short, this development approach is an applications-oriented approach, one that begins by 
establishing and prioritizing user requirements and seeks to add value to the information transfer 
process through the appropriate use of VR technology. 


INTRODUCTION 

This paper describes the development 
process used by GreyStone in the creation of 
virtual environments to support complex 
information transfer. Rather than focus on 
iterative improvements in display 
technology or image generators, what is 
presented here is a discussion of how the 
initial design of a virtual environment must 
be geared to the ultimate application of the 
system and how the various technology 
components that underlie that system are 
integrated to provide a working, value-added 
interface. The emphasis here is on 


recognizing the unique value of this medium 
for information transfer and maintaining 
product focus throughout the development 
process. This “applications-oriented” 
process is not specific to certain 
applications, but is adapted and modified 
depending on the requirements of the end- 
user and is driven by factors such as current 
and emerging capability, affordability, and 
the accomplishment of specific application 
objectives. The kinds of applications that 
have been developed using this process 
include planning, training, and entertainment 
experiences. 
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The Value of Virtual Reality 

First, it is useful to put this discussion in the 
proper context and provide some 
background. GreyStone is in the business of 
creating information processing and 
information transfer products. The company 
recognizes that information is a critical 
commodity - die commodity of the future. 
This focus on information is shaping the 
nature of business across the globe, and the 
possibilities for both providers and 
consumers is enormous. Today, the sheer 
volume of information available to 
consumers and businesses is staggering, and 
this level continues to increase 
exponentially. But, having access to a large 
volume of data is of no value unless the 
information is presented in a usable form. 
That is where virtual reality comes in: VR 
technology allows us to build systems that 
put information into a usable form and make 
it accessible to the user in an effective way. 

It is hard to underestimate the importance of 
information to human society. In fact, it 
could be argued that the ability to record and 
transfer information is the primary ability 
that distinguishes humans from all other 
animals [1 l Being able to represent ideas 
and observations as recorded symbols gave 
our early ancestors a major competitive 
advantage. And though other animals can 
use tools or communicate via language, only 
our species is able to capture important 
information in symbolic form to be passed 
on to future generations. So, the current 
information revolution is not the first, but 
one in a series of societal shifts that have 
influenced our evolution. The current 
“information age” is about the ability to put 
information into digital form, to process that 
data very quickly via computer, and to 
distribute information at the speed of light to 
everybody on the planet (and beyond). 


Information, then, is a representation of 
something else. In its representational form, 
it is inert, dependent on somebody or 
something to interpret its meaning and 
complete the transfer process. With the 
recent ability to rapidly create and copy 
information of high volume and complexity 
came the requirement to complete the 
information transfer process in more 
effective ways. The proper employment of 
VR technology allows this to be 
accomplished by presenting information at a 
lower level of abstraction than typically used 
by traditional computer interfaces. This 
means that a VR interface presents 
information in a form that is directly 
interpreted by the human senses, like 
images, sounds, and motion. Furthermore, 
these interfaces allow visible and invisible 
phenomena to be intermixed, correlated, and 
displayed. Such an interface may one day 
allow air traffic controllers, surgeons, and 
mission planners to better understand their 
complex, multi-dimensional problem spaces 
and allow them to make better decisions in 
less time. If the data representation and 
transfer process in a VR interface is accurate 
enough, the participant may get the sense of 
actually “being there”. This fact has led a 
number of developers to extend the 
application of VR to the entertainment and 
advertising markets. 

THE APPLICATIONS-ORIENTED 
DEVELOPMENT PROCESS 

Creating an effective virtual environment 
requires the coordinated participation of a 
multi-disciplinary team and the orchestration 
of that team throughout the many stages of 
the development process. To ensure that the 
resulting system achieves its expected aims, 
it is crucial that an applications-oriented 
approach be used during the project 
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definition phase, and that this definition then 
serves as a functional guide throughout the 
implementation process. A specific VR 
development project is used here as an 
example to illustrate points from the process 
description. Though any number of 
examples could have been chosen, this one 
demonstrates some unique requirements that 
had to be considered in the design phase. 
The example is an experience called Virtual 
Voyage ™, which was created for product 
promotion puiposes. In this experience, 
participants navigate a clipper ship from one 
port to another to deliver a cargo of whisky. 

Step 1 - Understand the User’s 
Requirements 

Perhaps the most critical step in designing 
an effective virtual environment is to put 
oneself at the end of the process and analyze 
how the system will be employed by the 
ultimate user. What are the crucial elements 
of information that must be present? What 
are the functions that the interface must 
provide? What is the background of the 
expected operators? Clear answers to these 
questions may indicate that a fully 
immersive environment is not desirable at 
all, which would have a drastic impact on 
the system design process. Other user- 
dependent questions will help frame the 
development process, like: How important 
is domain expertise in the creation of an 
effective interface? What is difficult about 
accomplishing this interface task using 
traditional techniques? What is the available 
budget? What kinds of ergonomic issues 
exist? By clearly addressing these sorts of 
questions during the project definition 
phase, the developed system takes on an 
applications-oriented purpose, helping to 
guide its evolution and ensure its usefulness. 


In the case of Virtual Voyage , the design of 
the system was driven not only by the 
general public that would eventually 
experience the system, but primarily by the 
customer: the advertising agency paying for 
the development of the system. Obviously, 
the advertiser’s goal is to feature the product 
in a relevant and entertaining way. Since the 
product being advertised is an alcoholic 
beverage, it was decided to create an 
experience from the days of prohibition, a 
recreation of the ferrying of liquor from the 
Bahamas to Long Island aboard the clipper 
ship of Captain McCoy. This theming of the 
experience is extremely important for giving 
the participants a context for involvement, 
and in this case, helping to achieve the 
advertiser’s goals. At this point, it becomes 
evident that a certain amount of domain 
experience will be required to ensure realism 
of the sailing vessel’s behavior and 
responses. With the purpose of the virtual 
environment clearly in mind, the entire 
experience can now be created around the 
established theme. 

Another important factor at this stage is a 
thorough understanding of the eventual 
participants who will experience the virtual 
environment. The users of the system are 
members of the general public, who happen 
to see this advertising event as it tours the 
country. It was desired to leverage the sense 
of immersion and interactivity with the 
environment that VR gives the participant, 
while maintaining the focus on die product 
being advertised. Since the users would 
likely experiencing VR for the first time, it 
was also important to keep the game concept 
straightforward and the interface simple. 
Further, the system must achieve its 
objective within just a few minutes of play. 
To achieve these aims and promote product 
identification, the general concept developed 
was for the participant to sail the virtual 
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vessel while protecting the cargo from 
various hazards along the way. These 
hazards could be in the air, on the water, or 
on the ship itself, so the visual interface 
would have to provide the participant with a 
full field of regard. A hand-held gun with a 
virtual representation would allow the 
participant to defend his or her cargo, 
thereby linking the participant with the 
scenario and building an identification with 
the product. 

Step 2 - Identify Where and How VR Adds 
Value 

When designing an interface for information 
transfer, one must be careful that the 
technologies chosen to support the interface 
actually enhance the transfer process rather 
than distract from it. The goal should be to 
effectively transfer the desired information, 
not showcase the latest VR product or 
technique. A big part of virtual environment 
development has to do with understanding 
how human beings detect and assimilate 
sensory information and what tools and 
techniques are available to reproduce these 
effects Of course, a certain amount of 
cost realism usually enters the equation, but 
even with compromises, a compelling, 
immersive environment can be achieved. 
The key requirements are to provide a high- 
fidelity visual scene, to correlate that visual 
scene with an appropriate soundscape, and 
to provide at least some level of somatic 
contact with the system (anything from a 
simple joystick interface to a motion-base), 
which also allows the participant to interact 
with the virtual environment. Additionally, 
this must all be accomplished in a small 
enough time frame that the user does not 
notice things like system latency or update 
rates. Though other sensory modalities can 
be brought into play, the cost is likely to 
outweigh the benefits. Again, the analysis 


should be based upon the requirements 
defined in step one. 

Having settled on the deck of an open 
sailing vessel as the environment to be 
recreated in Virtual Voyage , the primary 
requirement is to create a large and realistic 
visual scene. The fact that events in the 
environment are occurring all around the 
participant makes it a good candidate for a 
head-mounted display (HMD) with a head 
tracking system. Additionally, the visual 
resolution requirements in this case are 
lenient enough to allow an HMD to be used 
cost-effectively. HMDs are not always the 
best display medium, and their use certainly 
does not define the interface system as a 
virtual environment. Going back to the 
original guidelines for design, it was 
determined that a repeat monitor must be 
included to allow a larger audience to share 
what the participant sees within the HMD. 
This allows the information transfer process 
to reach a much larger audience. 

Somatic contact would be achieved through 
a physical mockup of the vessel’s helm and 
a plastic pistol. In fact, a portion of the deck 
and mast was eventually built to provide 
continuity of the theme between the real and 
virtual worlds. The design allowed the 
participant to actually steer the vessel by 
turning the wheel and observing the change 
in attitude relative to the prevailing wind in 
the sails and the motion of the waves. By 
way of a motion-tracker attached to the 
hand-held pistol, a virtual gun could be 
moved about the virtual space. These two 
modes of physical interaction were intended 
to get the participant’s body involved in the 
experience, helping to close the sense of 
presence. 

Finally, the auditory channel would have to 
be supported via a detailed sound profile of 



the dynamic environment. The interfaces 
chosen were headphones to transmit the 
soundtrack to the participant and 
loudspeakers to repeat those sound to the 
audience. To enhance the sense of 
immersion, a three-dimensional sound 
system was selected to allow the participant 
to localize sounds in the surrounding virtual 
space. Other sensory pathways could have 
been engaged, like wind blowing from the 
appropriate direction or a motion-base to 
simulate the toss of the ocean, but these 
were not included since the desired effect 
was achievable at a reasonable cost using the 
above components. 

Step 3 - Develop Concepts of Operation 

The last step before implementation can 
begin is to fully scope the operational 
employment of the virtual environment. 
Based on the preliminary definition 
established thus far, and keeping the user’s 
requirements in mind, the “flow” of system 
operation must be defined. For 
entertainment-oriented experiences, this 
would involve the development of an 
appropriate storyline, perhaps supported by 
artistic renderings of probable scenes and 
scenarios. More serious applications, like a 
mission planning system, would use sample 
scenarios to exercise the sequence of 
interactions during the planning process. 
Building on the basic goals and components 
established in steps one and two, the theme 
must now be “filled out” with details and 
complete concepts. 

In our example process, the operational 
sequence of Virtual Voyage was completed 
through the development of a few scenarios. 
An “objective” for the experience was 
defined: the participant would be scored 
based on the number of cases of scotch that 
were successfully delivered to the final port. 


The primary hazard to delivery would be a 
stowaway who would attempt to steal cases 
from the stack of cargo. This virtual thief 
could be (non-lethally) shot to discourage 
his troublesome activity. The stowaway 
would appear throughout each of the voyage 
scenarios. The scenarios involved 
navigating the vessel out of the natural 
harbor in the Bahamas (the initial scenario), 
and engaging opponents on the open ocean. 
During a storm scene, good sailing skills 
would reduce the loss of cargo over the side 
of the ship. On the way to “Gatsby’s 
Mansion” on Long Island, the participant 
would encounter a rival gang in speedboats 
who shoot at the cargo to reduce the player’s 
profits and an attack seaplane with a similar 
objective. By shooting these opponents, the 
participant ensures the safety of his or her 
cargo and thereby achieves a higher score. 

The choice of interface devices in step two 
has implications in the development of an 
operational concept. A detailed visual 
representation of the surrounding world 
meant that a substantial amount of physics 
would have to underlie the behavior of the 
graphical objects. This not only applies to 
the vessel being sailed by the participant, but 
to the computer-generated adversaries. Of 
course, attention would have to be given to 
the polygonal representation of the visual 
scene, the use of appropriate image textures, 
and the scene’s depth complexity. 
Additionally, each scenario would have an 
associate soundtrack so that objects and 
interactions were represented aurally. This, 
too, would be of high fidelity, with sound 
localization included. The kinds of sounds 
to be sampled and created (ocean waves, 
seagulls, creaking of the masts and rigging, 
gunshots, etc.) were identified at this point. 

Each of the scenarios or vignettes was 
defined in this way to serve as a 
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development template during the 
implementation phase. The final scenario 
was defined as the arrival at “Gatsby’s 
Mansion” (to some considerable fanfare, 
depending on the amount of cargo 
delivered). Following the initial departure 
scenario, the vignettes are chosen randomly 
to provide a sense of novelty for viewers 
who see the game played a few times. A 
consistent storyline was put together, and the 
timing of vignettes and the overall 
experience were established (30-45 seconds 
for each vignette and approximately 3 
minutes overall). 

Step 4 - Implementation and Integration 

Finally, the system must be constructed 
based on the foregoing design. Clearly, this 
is the bulk of the development process, 
though, unfortunately beyond the scope of 
this paper. If the project definition phase 
has been accomplished successfully, it will 
serve as a constant guide throughout the 
implementation process. Having established 
an applications-oriented goal for the virtual 
environment system, the following steps can 
be carried out to achieve project completion: 

• Artistic Rendering of Objects and Scenes 

• Simulation of Underlying Physics and 
Object Relationships 

• Modeling of Entity Behaviors and 
Interactions 

• Integration of the Human Participant with 
the Virtual Environment 


A photograph of a participant engaged in the 
completed Virtual Voyage experience is 
shown in Figure 1 and an image of the 
virtual environment itself is shown in 
Figure 2. 

CONCLUSION 

The development of a virtual environment is 
a complex undertaking requiring skills from 
many creative and engineering disciplines. 
To orchestrate these skills and keep the 
development process focused on the ultimate 
purpose of the information transfer system, 
it is important to adopt an applications- 
oriented methodology. This process consists 
of understanding the user’s breadth of 
requirements, recognizing the value of VR 
technology for the current application, 
applying the technology appropriately, and 
creating a detailed concept of operation prior 
to implementation. If such a methodology is 
followed during the design phase, the 
implementation task is more focused and 
directed, and the resulting system is more 
likely to achieve its objectives. 

REFERENCES 

1. Rothschild, Michael, “Cro-Magnons’ 
Secret Weapon”, Forbes ASAP, New York, 
September 13, 1993, pp. 19-20. 

2. Robinett, Warren, “Synthetic Experience: 
A Proposed Taxonomy”, Presence, 
Massachusetts, Volume 1, Number 2, Spring 
1992, pp. 229-247. 


62 








Figure 1. A participant immersed in the Virtual Voyage experience. 



Figure 2. The stowaway and speedboats harass the participant in Virtual Voyage . 
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